Cardiovascular Disease Detection using Ensemble Learning

One of the most challenging tasks for clinicians is detecting symptoms of cardiovascular disease as earlier as possible. Many individuals worldwide die each year from cardiovascular disease. Since heart disease is a major concern, it must be dealt with timely. Multiple variables affecting health, such as excessive blood pressure, elevated cholesterol, an irregular pulse rate, and many more, make it challenging to diagnose cardiac disease. Thus, artificial intelligence can be useful in identifying and treating diseases early on. This paper proposes an ensemble-based approach that uses machine learning (ML) and deep learning (DL) models to predict a person's likelihood of developing cardiovascular disease. We employ six classification algorithms to predict cardiovascular disease. Models are trained using a publicly available dataset of cardiovascular disease cases. We use random forest (RF) to extract important cardiovascular disease features. The experiment results demonstrate that the ML ensemble model achieves the best disease prediction accuracy of 88.70%.


Introduction
e human heart is the body's most important organ, responsible for pumping blood to every body region. e circulatory system comprises arteries, veins, and capillaries, all of which connect to the heart [1]. e heart is one of the body's most essential organs, yet it is vulnerable to illness and injury. Because of its danger to human life and the illnesses and injuries it causes, its presence cannot be ignored. Heart disease affects the heart's pumping systems, causing it to malfunction [2]. Cardiovascular disease is a category of ailments that are caused by heart problems. Cardiovascular disease may be diagnosed by shortness of breath, physical weakness, swollen feet, and exhaustion. High cholesterol, smoking, sedentary lifestyles, and high blood pressure are a few risk factors for cardiovascular disease [3]. e leading cause of mortality, according to the World Health Organization [4], is cardiovascular disease, which kills 18 million people a year, or about 32% of all fatalities.
e most common kind of heart disease is coronary artery disease. As a result, heart disease and stroke are major public health concerns. Clinicians often employ angiography to diagnose cardiovascular disease. In underdeveloped nations, where diagnostic technologies, physicians, and other resources are few, this diagnosis process is time-consuming and costly since it requires examining many variables. Heart disease has become one of the most critical medical topics in recent years as the mortality toll from the cardiovascular disease continues to climb. Prediction aids in the early detection of disease and the most effective treatment. e use of ML in medical diagnostics is proliferating. is may be partly ascribed to improving the categorization and identification of illnesses, giving data to help medical professionals discover and diagnose diseases, sustaining human health, and decreasing the mortality rate. When determining the likelihood of a disease incidence, ML classification approaches are often used [5,6]. is work's goal is a classification model that can predict cardiovascular disease.
Making predictions based on experience has become a central goal of ML research. e neural network is one of the most common methods of machine learning. is is the first step in supervised learning, in which a model is built using labeled data, and its performance is assessed using test data. Classification and regression difficulties are common in supervised learning.
is is the second form of learning, where the data is not labeled, and the model attempts to uncover the hidden patterns in the data. Data exploration results generate inferences about the nature of any concealed knowledge. Unsupervised learning is shown by the clustering method [7]. Reinforcement learning is the third form, which does not employ labeled data and does not link the conclusions to the data. An environment's intelligent agents [8] focus on this research. Use a real-world dataset of patients with cardiovascular illness to construct a classification model. is ML method is often used to predict the likelihood of illness occurrence [5,6]. To forecast the class of fresh samples, a model learned from training data is used [9,10]. Categorizing data into classes is also a supervised learning notion [10].
is work provides contributions to detecting cardiovascular disease more effectively and efficiently. Below are the major contributions of this paper.
(i) Proposed ensemble learning method for detecting cardiovascular disease including several classifiers in voting-based decision-making. (ii) Compare the proposed method to traditional machine learning approaches such as RF, K-nearest neighbor (KNN), multi-layer perceptron (MLP), and State-of-the-art research to determine the efficacy of the proposed approach. (iii) Experimental results show a 1.47% accuracy improvement compared to earlier state-of-the-art research and standard machine learning algorithms for the proposed approach.
Section 2 provides an overview of the relevant literature. Details regarding the proposed approach and the dataset are provided in section 3, and the outcomes and comparisons are provided in Section 4 in the comparative part. Finally, Section 5 concludes this work and provides a detailed description of future research.

Literature Review
Machine learning has sparked widespread interest in various fields, including health and medicine [11]. Researchers have proposed numerous viable approaches to detecting cardiac disease using machine learning. Consequently, studies on developing medical applications employing various machine learning algorithms and approaches have been published.
According to [12], machine learning may be used to measure and predict people's cognitive health using intelligent home sensors that have been built. A total of 179 individuals were viewed as volunteers by the respondents. We found an r � 0.79 association between the automated task quality evaluation ratings and the direct observation scores. Participants' cognitive health was predicted using machine learning approaches with an AUC of 0.64.
According to [13], machine learning may be used to detect cardiovascular illness based on a patient's clinical data. e RF model was the most accurate, with an accuracy rate of 86.60%. Motarwar et al. [14] developed an algorithmic system to predict heart disease risk. eir findings, at 95.08%, discovered that RF was the most accurate. Multiple algorithms, including Decision Tree (DT), NB, KNN, and RF, were used by Sabarish and Parvati [5]. According to the data, the KNN algorithm has the highest accuracy rate (90.7%). Additionally, a CNN model was used to detect images of Chinese herbal medication, achieving a 71% total accuracy. With an incredible 96.7% success rate, the usage of ANN for lung cancer detection was also found [15].
Some studies look at ways to improve the status quo and develop new classifiers. Using Natural Language Processing (NLP) to train and assess a depression prediction model, for example, was proposed in research by [16]. According to [17], a neural network was proposed for predicting diabetes, with a prediction accuracy of 87.3%. Compared to its baseline performance of 89% and 81%, the proposed approach enhanced the neural network's performance by 91% in the training and 86% in the testing.
It was hypothesized in [18] that the Particle Swarm Optimization (PSO) approach might be used to enhance a neural network's accuracy further. e research was based on a dataset that included 303 healthy and ill people cases.
ere were 72 features in the dataset, but only 13 of them were employed in the feature selection process using PSO. e PSO was utilized for feature selection and ranking after preprocessing the dataset. From a pool of thirteen options, ranking results indicated the eight most valuable options for improving neural network training accuracy using a feedforward backpropagation approach. According to the study's findings, the hybrid PSO-FFBP neural network was 91.94% accurate in predicting heart illness. It was also shown that diabetic retinopathy might be predicted using an upgraded version of the DNN using the Grey Wolf Optimization (GWO) algorithm and PCA. A questionnaire was developed by Nowshad et al. [19] in the Sylhet area of Bangladesh, where they visited many local hospitals and health care facilities to collect data. eir dataset has 564 occurrences and 18 characteristics. e SVM was the most accurate, with a 91% success rate.
Using machine learning approaches to identify key traits, Mohan et al. [20] have created a method for improving cardiovascular disease prediction accuracy. e proposed mixed RF and linear model method were 88.7% accurate in predicting heart disease. Au et al. [21] have presented a hybrid approach for diagnosing cardiac disease. Logistic regression was shown to have an accuracy rate of 89% when predicting heart disease. Authors in [22] proposed a hybrid model for predicting coronary artery disease. ey used three machine learning algorithms: RF, DT, and a mix of the two to create the model. e hybrid model was the most accurate, with an accuracy of 88.7%.
According to [23], they used standalone machine learning approaches to detect heart disease. ey used different ensemble learning models to improve the accuracy of a prediction model. ey used sleep disorder, stress mismanagement, and pollution factors as features.
For this study, we utilize data on cardiovascular disease. is dataset has around 70000 patients and 11 characteristics, which we believe is the third research to use. Also, we use various machine learning algorithms and deep learning algorithms to find the best one for predicting cardiovascular disease.

Proposed Approach
is section describes the methods to predict cardiovascular disease. Figure 1 depicts the proposed approach comprising six phases. e proposed approach begins with the selection of a dataset for the experiment. e Cardiovascular Disease dataset is used for experiments in this research. e preprocessing stage includes many steps that must be completed before model training. e features extraction approach is then utilized to determine the relevance of the features, and finally, several machine learning and deep learning classifiers are employed for experiments. Deep learning methods are also being evaluated in this study to detect cardiovascular disease. To identify the existence of cardiovascular disease, four prior ML classifiers are used: RF, KNN, DT, and Extreme Gradient Boosting (XGB). e ability of deep learning models on a particular dataset is tested using two deep neural network classifiers.

Dataset Selection.
e Kaggle repository's Cardiovascular Disease dataset (https://www.kaggle.com/datasets/ sulianova/cardiovascular-disease-dataset) is utilized in this research. ere are 70, 000 samples in the dataset and 13 attributes. 70% data is utilized for training models, while the remaining 30% is used to assess the model's performance. Table 1 shows the description of the features and explains each feature in-depth.

Data Preprocessing and Corelation Analysis.
Machine learning relies on data preprocessing to assess data quality and retrieves critical information that can influence the learning model's performance. Before training a model, preprocessing is essential. During the preprocessing stage, addressing diverse dataset characteristics, handling missing values, scaling, and standardization are dealt with during the preprocessing stage.
is includes managing small, huge, and noisy datasets, overcoming overfitting, dealing with class imbalance, and label encoding.
is work recommends using preprocessing approaches such as eliminating anomalies (outliers) and applying a standard scaler to the dataset to demonstrate the model's efficiency and acquire an acceptable and dependable accuracy for disease prediction. We employ a standard scaler to normalize the data within a predetermined range. It operates by altering characteristics like its distribution to reach a mean of 0 and a standard deviation of 1. Equation (1) defines a standard scaler, where S is the standardized form of z i .
ML models make better decisions depending on how labels are used.
e Label Encoder transforms the target labels into numeric forms so that machines can read them for improved disease diagnosis. We also made the following changes to the dataset to identify better the characteristics that most affect cardiovascular disease: (i) Body Mass Index (BMI) combines weight and height into a single characteristic that evaluates body fat percentage depending on height and weight. (ii) To make the dataset more understandable, features are modified while the information is preserved. e gender feature is converted to binary, and the age is translated from days to years in this dataset. (iii) Drop the dataset's year, height, weight, and id columns. (iv) Checking the maximum and minimum of quartiles to identify outliers and remove the rows that have outliers.
Pearson's coefficient (PCC) is used to get the corelationship between the characteristics to remove nonessential, duplicate, and redundant features from the data, as shown in Figure 2.
e correlation coefficient might be anywhere from −1 to 1. e features are negatively correlated if the value is close to −1; if the value is close to 1, the features are tightly coupled and considerably influence model performance. We set the threshold values to 0.85% to determine the PCC; if the correlation value is above the threshold, the feature is ignored; if the correlation value is below the threshold, the feature is kept. All characteristics are the same after the feature corelation analysis.

Feature Extraction.
It is necessary to select essential features that aid in classification. Its goal is to minimize dimensionality by identifying the most informative feature that may help the models perform better. We found that the RF feature selection approach identifies the essential features. It is also a prominent feature selection approach in machine learning methods, and it is worked well in realworld scenarios.

Classification Models.
We propose a machine learningbased ensemble model (XGB, KNN, and DT) and an ML and DL-based stacked model (ML ensemble model, DNN, and KDNN) to improve the classification results. We applied four machine learning methodologies for our experiment: RF, KNN, DT, XGB, and two deep learning models, DNN and KDNN. is section shows how to use the machine learning ensemble classifier and DL method to identify cardiovascular disease and evaluate our approach's success effectively.
Computational Intelligence and Neuroscience

Machine Learning Ensemble Classifier.
Researchers have used ensemble learning to solve a range of machine learning problems [24,25]. Because each dataset contains a wide range of problems, an ensemble learning approach is used to distinguish different diseases and categorize data appropriately. Every classifier in the ensemble estimates a class label whenever a new data point comes in. e class label that is predicted by the majority of the classifiers, or the class label with the most votes, is the one that is used for that instance. Try to improve performance by using a mix of MLclassifiers and different ways to vote [26]. ese strategies are better than the classic single-learning approach for better generalization results. e proposed ML ensemble classifier takes the predictions of several classifiers and uses a weighted majority method to develop the results. After each classification model has been fine-tuned, the best results are reported. e method in the equation is used to get the most votes in equation (2).
K-Nearest Neighbour (KNN) is the essential machine learning approach for regression and classification. e data is used in KNN computations, which utilize similarity measurements (e.g., distance function) to describe new points. e KNN calculation relies on the similarity of the items being compared. In KNN, classification is determined by the majority vote of its immediate neighbors. e class with the most neighbors is labeled on the data point [27]. As the number of nearest neighbors increases, the choice of k and the accuracy may improve. We used the default parameter settings given by the sklearn library.

Computational Intelligence and Neuroscience
Extreme Gradient Boosting helps to improve performance and use of memory resources. is ensemble learning approach uses XGBoost to improve the performance of classification. e XGBoost classifier is used to improve the accuracy of classification. XGBoost is a machine learning algorithm used on a large-scale dataset. It uses gradient descent to optimize the loss function and returns a prediction as a boosting ensemble of weak classification trees [28]. is model is unique in combining weak ML algorithms with an agile approach to learning. Gradient boosting is one of the few models of its sort. Gradient boosting uses the residual error to change the previous prediction to optimize the loss function. With XGBoost, the loss function is regularized to offer an objective function for assessing the model's performance, which is defined by Equation (3) indicates the parameters that may be inferred from the available data by adding oplus. e model's complexity is measured by the loss function (Loss) and the regularization term (Omega ′ ). e XGBoost algorithm parameters utilized in this research are as follows: e maximum depth parameter is used by the XGBoost model. 6 is the highest depth that may be reached. Booster is gbtree, eta value is 0.3 in addition to the following parameters: scale position weight is 1, min-children-weight is also 1, and finally, the booster is 1 and all other parameters remain default. Decision Tree may be used to sort categorical and numerical data alike. It resembles a tree in terms of structure. When working with medical data, DT is the most popular approach. Creating and analyzing a treeshaped graph is a cinch. As one of the most effective and extensively utilized methods for regulating learning, we turned to the DT classification method [29]. Building a stable decision tree for a specific data collection is straightforward.
is model also used the default parameters while training on the given dataset. Computational Intelligence and Neuroscience 5 network (RNN) models such as LSTM have also been employed, although LSTM did not perform well on the given dataset because of the abundance of numeric input. We used DL-based classifiers to evaluate cardiovascular disease detection and classification performance to provide a thorough study. e input layer with a ReLU activation function of 10 dimensions and a unit value of 16 is followed by three dense layers, each with a unit value of 12, 8, and 4, and a fullyconnected layer with a sigmoid activation function associated directly with a specific value that the model is trying to predict; this model is called the DNN model. A total of 176 parameters make up the input layer. is layer has 204 parameters, each with an individual value of 12. One hundred four parameters with eight units are set in the second hidden layer, and 36 parameters with four in the final hidden layer. e learning rate of the DNN model is 0.001, and the loss is calculated using Binary cross-entropy in the DNN model.
It is possible to learn all the parameters, and the proposed model performed well on the test data. To prevent overfitting, we used a neural network known as KDNN, a robust medical disease detection approach. When we reduced the size of the dataset by removing outliers, we applied an effective medical disease detection Neural Network (KDNN) to control overfitting. e Keras Deep Neural Network (KDNN) model employs an Adam optimizer with Binary cross-entropy to compute the loss and simplify finetuning. e proposed KDNN model has a single input layer as its architecture. Finally, a single, fully connected layer is revealed after many hidden layers. Finally, a single, fully connected layer is revealed after many hidden layers. e 132 parameters are presented in the input layer. One hundred thirty parameters are buried in the following layer. e third hidden layer has 88 parameters, while the final hidden layer has 54 parameters with 411 training parameters. is fine-tuned KDDN model is successfully trained and tested on the Cardiovascular Disease dataset.

Results and Discussion
is section briefly describes the results and compares the baseline methods. is study's major objective is to examine the classification performance of recommended machine learning and deep learning classifiers for cardiovascular disease detection. e experiments for this study are done with the Cardiovascular Disease dataset. Both machine learning and deep learning are used in this study. e experiments are done in two different ways. Machine learning models are used in the first step to getting better results. We devised an ML-based ensemble model that performed well on the given dataset. is is done to get more reliable results. In the second step, we conducted experiments with deep learning approaches. Because there is not much data to work with, the deep learning models did not do well, so we made a stacked ensemble model to get better results from deep learning. is study used several measures of performance: accuracy, precision, recall, F1 score, confusion matrix, and area under the ROC curve (AUC-ROC score). e dataset is split into two parts: 70% of the data is used to train the model, and 30% is used to test the model. Python 3 and a Jupyter notebook running on an i5-8300H laptop with 16 GB RAM and a GTX 1050 2 GB VRAM are used to simulate the proposed model. Table 2 demonstrates the results of the machine learning classifiers on the Cardiovascular Disease dataset to identify the existence of cardiovascular disease. RF, KNN, DT, and XGB are employed in this research. Table 2 demonstrates that, compared to other machine learning models, the RF classifier obtained the best accuracy rate of 88.65%. RF's precision, recall, F1, and AUC scores are 90.03%, 80.03%, 80.02%, and 92%, respectively. We also plot the confusion matrix of the RF classifier.    Table 3. Both the deep learning models performed exceptionally well. Both models' accuracy score is almost the same but less than the machine learning model because the deep learning model performed well on a large amount of data. e DNN model gets the highest accuracy than the KDNN model. e accuracy, precision, recall, F1 score, and ROC AUC score of the DNN model are 87.59%, 97.77%, 76.27%, 65.54%, and 91.50%. We also computed the loss of the DNN model, which is 30.19%. e training and validation loss of the DNN model is shown in Figure 5 while Figure 6 shows the training and validation accuracy of the DNN model on 400 epochs.   Figure 7, and the ROC AUC curve plot is shown in Figure 8.

Experimental Results.
A machine learning ensemble classifier and an ML and DL stacked classifier are used to get the final findings in this study.
is classifier has the greatest accuracy of 88.70% among all other classifiers. However, the lack of data meant that the stacked classifier did not perform effectively. ML Ensemble has an 88.02% precision and recall rate, an 88.01% F1 score, and a 93% ROC AUC. e confusion matrix and AUC ROC of the proposed ML Ensemble classifier are shown in Figures 9 and 10. e results show that the proposed ML ensemble classifier can efficiently detect the presence of cardiovascular disease from the Cardiovascular Disease dataset.

Comparative Analysis.
e comparative analysis of the proposed and baseline approaches is presented in Table 5. We broadly compare with two baseline research studies [30,31]. e experimental setting of the proposed and baseline studies are almost the same. is research used an ML Ensemble method to detect cardiovascular disease.
In Table 5, we compare this study's findings to our baseline method. Alfaidi et al. used various ML models (LR, RF, DT, NB, KNN, SVM, and MLP) and achieved the highest result from the MLP model with 87.23% accuracy [30]. Arroyo and Delima used two Artificial neural networks, and five ML models (LR, DT, RF, SVM, and KNN) [31]. is study obtained the highest accuracy using the GA-ANN model, 73.43%. e proposed approach used a majority voting approach and created an ML Ensemble model, which obtained the highest accuracy compared with the baseline results. e ML Ensemble model obtained an accuracy score of 88.70%. e accuracy gain of 1.47% using the ML Ensemble model shows that the proposed ML Ensemble performed efficiently to detect cardiovascular disease.

Conclusion
is study proposed ensemble-based machine and deep learning approaches to predict cardiovascular disease. For this study, we used 70000 patients' data with various forms of cardiovascular disease. e accuracy of the models was used as a measure of their performance. In addition, we selected additional informative characteristics that influence the models' performance. According to the results, the ML Ensemble model was the most accurate in predicting cardiovascular disease. We have added a few procedures to get the dataset ready for analysis. We may employ various strategies to find the best features for our future work. For a better and more accurate assessment, additional datasets may be employed. Finally, deep learning and reinforcement learning approaches may tackle the prediction issue to detect cardiovascular disease more efficiently.

Data Availability
e Cardiovascular Disease data used to support the findings of this study are included in the article.

Conflicts of Interest
e authors declare that they have no conflicts of interest.